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ABSTRACT 

Data  density  has  a  crucial  impact  on  the  accuracy  of  Digital  Elevation  Models  (DEMs).  In  this  study,  DEMs  were 
created  from  a  high  point-density  LIDAR  dataset  using  the  bare  earth  extraction  module  in  Quick  Terrain  Modeler. 
Lower  point-density  LIDAR  collects  were  simulated  by  randomly  selecting  points  from  the  original  dataset  at  a  series 
of  decreasing  percentages.  The  DEMs  created  from  the  lower  resolution  datasets  are  compared  to  the  original  DEM. 
Results  show  a  decrease  in  DEM  accuracy  as  the  resolution  of  the  LIDAR  dataset  is  reduced.  Some  analysis  is  made 
of  the  types  of  errors  encountered  in  the  lower  resolution  DEMs.  It  is  also  noted  that  the  percentage  of  points 
classified  as  bare  earth  decreases  as  the  resolution  of  the  LIDAR  dataset  is  reduced. 
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1.  INTRODUCTION 

As  part  of  the  ISPRS  Working  Group  III,  a  study  completed  by  George  Sithole  and  George  Vosselman  looked  at  the 
performance  of  a  variety  of  bare  earth  extraction  algorithms. 1  One  of  the  stated  purposes  of  this  study  was  “to 
determine  the  performance  of  filtering  algorithms  under  varying  point  densities”  in  order  to  understand  the  trade-offs 
between  cost  efficiency  and  DEM  accuracy.  This  balance  between  cost  and  resolution  is  still  relevant  today. 

A  portion  of  the  data  and  ground  truth  information  from  the  Sithole/Vosselman  study  was  used  to  complete  the 
analysis  presented  here.  A  second  dataset  collected  over  Sequoia  National  Park  in  August  2008  was  used  to  perform  a 
qualitative  analysis  of  the  effects  of  LIDAR  point  density.  Ground  truth  data  was  not  available  for  the  Sequoia  dataset. 

2.  THE  LIDAR  DATA 

The  data  used  in  the  Sithole/Vosselman  study  is  available  for  download  from  http://www.itc.nl/isprswgIII- 
3/filtertest/index.html.  This  site  also  contains  further  information  about  the  study  design  and  results.  The  datasets 
were  collected  over  the  Vaihingen/Enz  test  field  and  Stuttgart  city  center.  A  subset  of  these  datasets  containing  mostly 
vegetation  features  were  used  here.  The  area  of  each  subset  is  between  0.5  and  1.0  km2.  See  Figure  1. 

According  to  the  Sithole/Vosselman  metadata,  the  average  point  spacing  of  the  rural  and  forested  sites  was 
approximately  2.0  to  3.5  meters  between  points  (0.18  points  per  square  meter). 1  The  Quick  Terrain  Modeler  software 
has  a  statistics  function  that  gives  the  average  point  spacing  of  the  dataset,  and  this  measure  gave  a  slightly  smaller 
value  (1.5  to  1.6  meter  spacing)  on  average  for  the  rural/forest  sites.  The  Quick  Terrain  Modeler  measurement  was 
used  throughout  to  maintain  consistency. 

The  Sequoia  National  Park  dataset  was  collected  by  Airborne- 1  Corporation  in  August  of  2008.  The  average  point 
spacing  of  the  subset  used  for  this  study  was  3.36  points  per  square  meter.  The  area  covered  by  this  dataset  is 
approximately  5.8  km2.  See  Figure  2. 
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Fig.  1.  Outlines  of  Sithole/Vosselman  subsets  overlaid  on  imagery  using  Google  Earth,  (a)  FSite5,  (b)  FSite6,  (c)  FSite7, 
and  (d)  FSite8. 


Fig.  2.  (a)  Sequoia  National  Park  dataset  as  displayed  by  Quick  Terrain  Modeler,  with  coloration  according  to  point 
elevation,  (b)  Outline  of  Sequoia  National  Park  data  overlaid  on  imagery  using  Google  Earth. 
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3.  METHODS 


The  bare  earth  extraction  algorithm  in  Quick  Terrain  Modeler  is  specifically  designed  to  work  over  forested  areas.  For 
this  reason,  four  (of  eight  available)  Sithole/Vosselman  subsets  collected  over  rural  landscapes  and  forested  areas  were 
chosen  for  the  analysis.  A  5.8  km2  subset  of  the  Sequoia  National  Park  dataset  was  also  chosen.  For  each  of  the 
datasets,  13  lower  resolution  point  clouds  were  created  by  randomly  selecting  points  from  the  original  dataset  at  a 
series  of  resolutions  (90%,  66%,  50%,  30%,  10%,  5%,  3%,  1%,  0.5%,  0.3%,  0.05%,  0.03%,  and  0.01%). 

Quick  Terrain  Modeler  is  a  powerful  analytical  tool  developed  at  the  Johns  Hopkins  University’s  Applied  Physics 
Laboratory  for  working  with  3-D  data,  including  “LIDAR,  SAR,  multibeam  sonar,  DEM,  and  other  data.”2  The 
software  enables  the  user  to  import  many  different  types  of  data,  building  models,  many  visualization  and  editing 
tools,  and  a  suite  of  analysis  tools. 

The  bare  earth  extraction  module  of  Quick  Terrain  Modeler  can  classify  each  point  in  the  original  point  cloud  as  either 
belonging  to  ‘cloud’  or  ‘surface’.  The  module  can  also  create  a  gridded  surface  file.  Both  of  these  outputs  were  used 
to  analyze  the  effects  of  reducing  the  point-cloud  resolution. 

There  are  several  parameters  that  can  be  adjusted  in  the  Quick  Terrain  Modeler  software  to  give  a  better  bare  earth 
extraction  result  if  the  user  knows  something  about  the  dataset  and  the  desired  product.  To  maintain  consistency,  and 
to  focus  mainly  on  the  effects  of  reducing  the  resolution  of  the  point  cloud,  the  only  parameter  that  was  adjusted  as  the 
DEMs  were  created  was  the  ‘scale’  of  the  surface  model.  The  ‘scale’  refers  to  the  average  spacing  between  points  to 
create  the  gridded  surface  file. 2  This  value  was  set  to  match  the  average  spacing  of  the  point  cloud  as  measured  by  the 
Quick  Terrain  Modeler  statistics  function. 


4.  RESULTS 

The  percentage  of  points  classified  as  bare  earth  was  determined  for  each  of  the  datasets.  As  shown  in  Figure  3,  the 
percentage  of  points  classified  as  bare  earth  decreases  as  the  overall  number  of  points  in  the  data  cloud  is  reduced. 
Since  the  phenomenon  occurs  for  all  the  datasets  analyzed,  we  are  confident  that  this  is  a  function  of  the  type  of  bare 
earth  extraction  algorithm  being  used. 


Fig.  3.  The  percentage  of  points  classified  as  bare  earth  using  the  Sithole/Vosselman  data  (FSite5,  FSite6  and  FSite7), 
and  the  Sequoia  National  Park  data. 


The  DEMs  created  for  the  Sequoia  National  Park  dataset  are  shown  below  in  Figure  4.  Similar  files  were  created  for 
each  of  the  Sithole/Vosselman  subsets  as  well. 
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DEMs  created  from  Sequoia  National  Park  LIDAR  data 

-  Percentage  of  original  dataset 

-  Number  of  points  classified  as  bare  earth  /  Number  of  points  in  data  cloud 

-  Average  spacing  between  points  in  data  cloud _ _ _ 

50% 

338,477/  1,524,477 
0.8018  m 


100% 

834,037/3,051,400 
0.5458  m 


90% 

681,935/2,742,629 
0.5782  m 


66% 

474,583  /2,016,401 
0.6865  m 


1% 

4,506/30,326 
7.08  m 


0.5% 

2,473  /  15,241 
10.4278  m 


0.3% 

1,243  /9,087 
13.616m 


0.05% 

66/  1,537 
35.1141  m 


0.03% 

23  /931 
48.4749  m 


0.01% 
0/308 
69.0988  m 


Fig.  4.  DEMs  created  from  Sequoia  National  Park  LIDAR  data 
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5.  ANALYSIS 


A  qualitative  assessment  of  the  DEMs  created  shows  the  level  of  detail  in  the  DEMs  dropping  off  as  the  resolution  of 
the  dataset  decreases.  This  is  expected,  and  there  are  no  glaring  errors  due  to  the  decreasing  resolution.  When  the 
resolution  of  the  dataset  is  decreased  so  that  the  average  point  spacing  between  points  exceeds  60  meters,  the  Quick 
Terrain  Modeler  bare  earth  extraction  algorithm  does  not  find  any  bare  earth  points,  and  all  points  are  classified  as 
‘cloud’. 

A  quantitative  assessment  was  completed  for  areas  with  corresponding  reference  data,  which  includes  subsets  of  the 
Sithole/Vosselman  datasets.  The  reference  data  consists  of  lists  of  points  from  the  original  dataset  that  are  known  to 
be  bare  earth  surface  points.  These  points  were  generated  by  manually  filtering  the  data,  making  use  of  landscape 
information  and  aerial  imagery  where  available. 1  The  reference  data  covers  only  small  portions  of  the  original 
datasets,  and  were  chosen  to  highlight  some  of  the  areas  expected  to  be  difficult  for  bare  earth  extraction.  These  sites 
include  data  gaps,  steep  slopes,  bridges,  and  buildings. 

The  reference  files  for  FSite5  (samples  5.1,  5.2  and  5.3)  contain  74,697  points,  and  cover  an  area  totaling  0.09  km2. 
Sample  6.1  contains  35,060  points  originally  from  FSite6,  and  covers  an  area  of  approximately  0.05  km2.  Sample  7.1 
has  50,705  points  originally  from  FSite7,  and  covers  a  spatial  area  of  0.02  km2.  There  was  no  reference  data  available 
for  FSite8.  Likewise,  there  was  no  reference  data  available  for  the  Sequoia  National  Park  data. 

Truth  tables  were  created  for  each  of  the  DEMs  to  tabulate  the  accuracy  of  the  bare  earth  extraction.  The  errors 
occurring  in  each  DEM  can  be  classified  in  two  ways,  as  defined  in  the  Sithole/Vosselman  study. 1  A  Type  I  error 
refers  to  the  case  where  a  bare  earth  point  is  classified  as  a  cloud  point.  A  Type  II  error  occurs  when  a  cloud  point  is 
classified  as  a  bare  earth  point. 

Graphical  representations  of  the  information  available  in  each  of  the  truth  tables  are  displayed  below.  Accuracy  is 
measured  by  dividing  the  total  number  of  correctly  classified  points  by  the  total  number  of  points  in  the  reference  files. 


Fig.  5.  The  accuracy  of  the  DEM  results  decreases  as  the  resolution  of  the  point  cloud  is  reduced. 


Proc.  of  SPIE  Vol.  7323  732301-5 


Downloaded  from  SPIE  Digital  Library  on  20  Aug  2010  to  205.155.65.226.  Terms  of  Use:  http://spiedl.org/terms 


Type  I  Errors 
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Fig.  6.  Percentage  of  points  classified  incorrectly  as  Type  I  errors  (points  incorrectly  classified  as  ‘cloud’). 


As  seen  in  Figure  3,  the  percentage  of  points  classified  as  bare  earth  decreases  as  the  overall  number  of  points  in  the 
cloud  is  reduced.  This  could  explain  why  the  number  of  Type  I  errors  (missed  ‘ground’  points)  increases  as  the 
resolution  of  the  dataset  is  reduced. 
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Percentage  of  points  classified  incorrectly  as  Type  II  errors  (points  incorrectly  classifed  as  ‘ground’). 


An  attempt  was  made  to  determine  the  source  of  the  Type  I  and  Type  II  errors.  Unfortunately,  because  of  the  very 
large  percentages  of  points  incorrectly  classified  as  ‘cloud’  (Type  I  errors),  it  was  hard  to  determine  the  source  of  these 
errors.  These  points  are  displayed  in  green  in  the  figures  below.  There  were  fewer  Type  II  errors  (points  being 
incorrectly  classified  as  ground);  these  points  are  shown  in  red.  From  looking  at  these  images,  it  seems  that  the  bare 
earth  algorithm  might  classify  LIDAR  points  occurring  on  steep  slopes  or  structures  incorrectly  as  ‘ground’. 
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(c)  Sample  5.3 


(e)  Sample  6.1  (f)  Sample  7.1 

Fig.  8.  3-D  renderings  of  incorrectly  classified  points.  Green  represents  points  that  were  incorrectly  classified  as 

‘cloud’  (Type  I  errors),  and  red  represents  points  that  were  incorrectly  classified  as  ‘ground’  (Type  II  errors). 
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6.  FUTURE  WORK 


Other  LIDAR  software  packages  are  available  that  provide  a  bare  earth  extraction  capability.  It  would  be  useful  to 
compare  different  algorithms  and  determine  which  effects  are  due  to  the  bare  earth  extraction  algorithm,  and  which  are 
a  function  of  reducing  the  density  of  the  LIDAR  dataset. 

A  more  in-depth  quantitative  assessment  can  be  completed  with  sufficient  ground  truth  information  for  the  Sequoia 
National  Park  dataset.  With  the  higher  resolution  of  this  dataset,  we  would  expect  that  the  accuracy  of  the  DEM 
classification  result  will  improve.  This  would  allow  a  more  thorough  investigation  of  the  sources  of  Type  I  and  Type 
II  errors. 
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